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Abstract 


This report summarizes how school and district leaders and academic coaches in two Texas school 
districts used assessment and other types of data to assess the quality of teaching and learning, to 
coach and supervise teachers, and to guide management decisions. The report also describes how 
district and school leaders supported teachers' use of data. The results in this report are based on 
interviews with district and school leaders and academic coaches, supplemented by observations 
of feacher team meetings. The data most frequently used were from three- or nine-week tests and 
from classroom observations. School leaders also reported using data from non-assessment sources 
such as attendance, discipline referrals, and surveys to intervene with students and adjust school 
procedures. 
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Introduction 

As school systems become increasingly data-rich— with longitudinal student information 
systems providing electronic reports on student test scores, grades, attendance, discipline, 
course participation, course credit, academic interventions, dropping out, and graduation- 
educators have an opportunity to use this windfall of data to improve teaching and learning 
in their schools (Datnow, Park, & Wohlstetter, 2007; Wayman, Spring, Lemke, & Lehr, 

2012). This case study examines the use of data in two school districts with well-developed 
student information and reporting systems. The purpose of the study is descriptive: to 
describe how educators in the two districts use different types of data to assess teaching 
and learning, to coach and supervise teachers, and to guide instructional and management 
decisions. 

The use of data is part of a larger set of practices and policy actions intended to improve 
outcomes for students. Many of these practices are listed in the Core Practice Framework 
(ACT, 2012), which divides educator practices related to teaching and learning into five 
areas of focus, or themes: 

1. Curriculum and Academic Goals— What do we expect all students to know and be 
able to do in each course, grade, and subject? 

2. Staff Selection, Leadership, and Capacity Building— How do we select and develop 
the leaders and teachers needed to ensure every student in the system is able to meet 
these expectations? 

3. Instructional Tools: Programs and Strategies— What programs, strategies, materials, 
and time allocation do we use to teach the necessary content and skills? 

4. Monitoring Performance and Progress— How do we know if students are learning 
what they should? 

5. Intervention and Adjustment— If students are not learning what they should, what do 
we do about it? 

The practices assign responsibility for related tasks to district leaders, school leaders, and 
classroom teachers. For example, with regard to the use of data (Theme 4), district leaders 
might be in charge of district-wide benchmark assessment and data systems; school 
leaders might arrange for regular meetings among teachers to discuss the assessment 
results; and classroom teachers might use the results to modify instruction in their 
classrooms. Rubrics are available in the Framework documentation so that educators can 
self-assess their degree of implementation of each practice. The practices are closely 
interconnected across themes: for example, teacher training and teacher collaboration 
(Theme 2) are essential for teachers’ effective use of data to monitor student performance 
(Theme 4) and to select and use appropriate student interventions (Theme 5). 
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Data use may be thought of in terms of the types of data, the users, and the uses to which they put 
the data. Types of data examined in this study include: 

■ Test data 

~ District-wide benchmark tests given every six or nine weeks 
~ School-specific common assessments given every three weeks 
~ State accountability tests 
~ College readiness tests and their precursors 
~ Other standardized assessments 

■ Other types of data 1 

~ Classroom observations 
~ Grades 
~ Attendance 
~ Discipline 
~ Course completion 
~ Graduation, transfer, and dropout data 
~ Surveys of students, teachers, or parents 

Users include audiences inside the school system such as students, classroom teachers, academic 
coaches, and school and district administrators; and outside audiences such as parents, school 
board members, and community leaders. This study looked at data use by teachers, academic 
coaches, and school and district administrators. 

Uses of data by educators at the district, school, and classroom levels may be put into a number of 
categories: 

1 . To identify individual student needs and place students in groups, interventions, programs, and 
classrooms. Data may be used to monitor overall student progress; to customize learning 
opportunities for individual students; to place students in small learning groups or short-term 
intervention programs; to place students in classrooms or academic courses; and to assign them 
to or exit them from programs such as bilingual or special education programs and programs 
for gifted and talented students (Breiter & Light, 2006; Coburn & Talbert, 2006; Light, Honey, 
Heinze, Brunner, Wexler, Mandinach, & Fasca, 2005; Marsh, Pane, & Hamilton, 2006; Schildkamp 
& Kuiper, 2010; Wayman, Cho, Jimerson, & Spikes, 2012; Wayman, Spring, Lemke, & Lehr, 2012; 
Young, 2006). 

2. To modify curriculum and instruction. Data may be used to identify learning objectives that 
students didn’t learn and that must therefore be retaught; to identify objectives already mastered 
by students; to address gaps in students’ prerequisite knowledge and skills; to modify the 
sequence of topics; to adjust the amount of time allocated to each topic; and to monitor whether 


1 Data types that pertain to students may also be classified by whether they measure achievement (accumulated knowledge and skill), 
behaviors, or goals— roughly corresponding to the cognitive, psychosocial, and career development domains cited in ACT (2007). 
Tests generally measure achievement; attendance and discipline data reflect student behaviors; while grades, course completion, and 
graduation capture a mix of behavior and achievement Standards-based grading and graduation testing requirements are attempts 
to increase the achievement component of those measures. Goals may be captured in surveys on students’ post-graduation plans. A 
fourth information category, attitudes, may be captured in survey data. 
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the district's curriculum is being taught to the desired level of rigor (Breiter & Light, 2006; 

Coburn & Talbert, 2006; Datnow, Park, & Wohlstetter, 2007; Kerr, Marsh, Ikemoto, Darilek, & 
Barney, 2006; Light, Honey, Heinze, Brunner, Wexler, Mandinach, & Fasca, 2005; Marsh, Pane, & 
Hamilton, 2006; Schildkamp & Kuiper, 2010; Wayman, Cho, Jimerson, & Spikes, 2012; Wayman, 
Spring, Lemke, & Lehr, 2012). 

3. To motivate students and educators. Data can be used to set goals for individual students, 
classrooms, and schools; to monitor whether these goals have been met; and to recognize 
individual and group success (Breiter & Light, 2006; Datnow, Park, & Wohlstetter, 2007; 

Hamilton, Halverson, Jackson, Mandinach, Supovitz, & Wayman, 2009; Light, Honey, Heinze, 
Brunner, Wexler, Mandinach, & Fasca, 2005; Marsh, Pane, & Hamilton, 2006; Schildkamp & 
Kuiper, 2010; Wayman, Cho, Jimerson, & Spikes, 2012; Wayman, Spring, Lemke, & Lehr, 2012). 

4. To coach and supervise teachers and other school personnel. Data may be used to guide 
discussions with or among educators; to inform educators of their strengths and to assist them 
with their weaknesses; and to identify individuals for possible promotion or termination (Breiter 
& Light, 2006; Datnow, Park, & Wohlstetter, 2007; Light, Honey, Heinze, Brunner, Wexler, 
Mandinach, & Fasca, 2005; Schildkamp & Kuiper, 2010). 

5. To adopt and evaluate programs and management decisions. Data can be used to pilot new 
programs or evaluate old ones; to adjust school routines, procedures, and schedules by 
troubleshooting difficulties with student attendance or behavior; and to examine whether past 
decisions have had the desired consequences (Coburn & Talbert, 2006; Luo, 2008; Marsh, Pane, 
& Hamilton, 2006; Schildkamp & Kuiper, 2010). 

6. To communicate information to outside audiences. Data can be used to communicate with parents 
when their children need additional academic or behavioral assistance; and to communicate 

with parents and public audiences about school programs, activities, and performance (Coburn 
& Talbert, 2006; Light, Honey, Heinze, Brunner, Wexler, Mandinach, & Fasca, 2005; Luo, 2008; 
Schildkamp & Kuiper, 2010; Wayman, Cho, Jimerson, & Spikes, 2012; Wayman, Spring, Lemke, & 
Lehr, 2012). 

Research Questions 

We sought to address four basic questions in this study: 

1 . What data did district and school leaders collect to determine the quality of teaching and learning 
in the school(s) under their supervision, and how did they use the data? 

2. What support did district leaders provide to facilitate the use of data by teachers and school 
leaders? 

3. What support did school leaders provide to facilitate the use of data by teachers and students? 

4. What information was discussed in teacher team meetings and how was it interpreted? 
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Methodology 


Participating Districts 

Two Texas school districts participated in this study. Both districts were known to be data-rich, with 
longitudinal student information systems providing electronic reports on student test scores, grades, 
attendance, discipline, course participation, course credit, academic interventions, dropping out, and 
graduation. In addition, the leadership teams were known to be interested in promoting effective 
data use and open to hosting a study of data use. 2 Both districts serve a majority-disadvantaged 
student population. Table 1 contains information on the two districts' demographics. 

Table 1 Student Enrollment and Demographics in the Participating Districts 



District A 

District B 

District size category 
(student enrollment) 

25-75,000 

5-25,000 

Percent economically disadvantaged* 

> 75% 

50-75% 

Percent African American 

0-10% 

10-25% 

Percent Hispanic 

> 75% 

40-50% 

Percent ELL 

> 25% 

5-25% 


‘“Economically disadvantaged" is defined by eligibility for the free and reduced-price school lunch program. 


Data Collection 

One researcher visited a total of three elementary schools (K— 4), two middle schools (5-6), two 
intermediate schools (7-8), and two high schools (9-12) in the two districts, interviewing school 
leaders, academic coaches, and district central office personnel (Table 2). The schools were selected 
by the research director in District A and the chief academic officer in District B based on school 
leaders' willingness to participate. Interviews took place in the fall of 2012. 

Table 2 School and District Personnel Interviewed 

District A District B 

District administrators (4) 3 District administrators (3) 

School leaders (6) School leaders (5) 

Academic coaches (6) 


2 Knowledge about the two districts was acquired by staff of the ACT Learning Management Team through a history of conducting 
trainings in the two districts and working with members of their leadership teams. 

3 Numbers in parentheses represent the number of people in each position who were interviewed. 
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“District administrators" included a variety of central office positions, including directors of academic 
programs, curriculum, and dafa training and support. School leaders were principals in District B, 
and either principals or assistant principals in District A. Academic coaches were non-supervisory 
staff responsible for working with teachers to improve instruction. The majority of the interviews 
were one-on-one. Lists of the interview questions for district and school leaders are contained in 
Appendix A. The academic coaches were asked the “school leader” questions. 

In addition, the researcher attended and recorded a number of teacher and administrator team 
meetings in the two districts (Table 3). 


Table 3 School and District Team Meetings Attended 


District A 

School-Level Teacher Team Meetings 


District B 

District-Wide Team Meetings 


2nd-grade mathematics 
3rd-grade mathematics 
5th-grade reading 
7th-grade mathematics 
7th-8th-grade science 
9th-grade English 
9th-grade biology 


PreK-K English language arts 
3rd-grade social studies 
Assistant principals 
Curriculum coordinators 


Data Analysis 

The interviews and meeting proceedings were transcribed by a professional transcriber and coded 
with keywords by the researcher using Atlas.ti software, so that all quotes pertaining to a particular 
topic could be easily extracted. A list of coded keywords is contained in Appendix B. Keywords were 
chosen in part based on the interview questions, and in part based on themes that appeared in the 
transcripts. Many quotations from the interviews were coded with multiple keywords. The researcher 
then selected keywords associated with each interview question and used the software to compile 
all quotations coded with those keywords. For example, all quotations associated with the keyword 
“district benchmarks" (referring to district benchmark assessments) were extracted into a single file. 
A review of these sets of quotes produced a general picture of how district benchmark assessment 
data are used in the two districts, the prevalence of the use of the data by the interviewees, and 
how district and school leaders support the use of the data. From each keyword-associated file, the 
researcher summarized the information contained in the transcripts and selected quotes for inclusion 
in the paper that appeared to be illustrative of specific ways that benchmark assessment data are 
used. 

Limitations 

This study describes how some practitioners in two districts reported that they use data and also 
draws on observations of a small number of teacher meetings in the two districts in which data 
were discussed. We are limited in our ability to generalize from these reports and observations to 
how others in the same district or in other districts across the nation are using data. The practices 
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reported were based on the accounts of a relatively small sample selected by their districts, and 
may not be representative of what a larger random sample of interviewees from those districts or 
other districts might have described. As mentioned earlier, the districts were selected based on their 
relationship with trainers from ACT. We would expect educators in these districts to be more active 
data users than average. Also, the researcher did not directly observe many of the data-use practices 
described in the interviews. Nor did we collect practice information from a large enough number 
of schools or districts to examine correlations between their practices and school and district 
performance indicators. 


Results 

This section is organized around the four research questions: (1) the types of data that school and 
district leaders reported that they use and how they used each type of data; (2) how district leaders 
supported the use of data by teachers and school leaders; (3) how school leaders facilitated data 
use by teachers and students; and (4) the ways that data were discussed in teacher team meetings. 

What data did district and school leaders use to assess the 
quality of teaching and learning in the school(s) under their 
supervision, and how did they use the data? 

District Benchmark Assessments. Curriculum-aligned benchmark tests, given every six or nine 
weeks in the four core areas of English language arts, mathematics, science, and social studies, 
provided a major source of information that district and school leaders in the two districts used to 
monitor student learning. 

District A’s benchmark exams, the Curriculum-Based Assessments (CBAs), were given every nine 
weeks in grades 2-6 and every six weeks in grades 7-11 . The district tested reading, mathematics, 
and science beginning in grade 2, writing was added in grades 3 and 4, and social studies was 
added to the other four subjects in grades 5-8. In high school, curriculum specialists or experienced 
teachers working in collaboration with those specialists wrote the CBAs for district-wide use for 
specific courses, such as Biology I or Algebra II. In high school English, there were no district-wide 
CBAs, but teachers at each school wrote their own common six-week assessments. In first grade, 
teachers administered district-written CBAs in mathematics, reading, and writing in the spring, but 
the results were not entered into the district’s central electronic data system. 

In District B, teachers administered benchmark exams written by curriculum specialists every 
nine weeks in kindergarten through high school in the four core subject areas of reading/writing, 
mathematics, science, and social studies. As in District A, high school benchmarks were written for 
specific courses in the four core subject areas. In both districts, teachers reviewed the benchmark 
exams before they are given in order to ensure that their content aligned with what was actually 
being taught during the six- or nine-week period: 

The district specialist writes the district benchmark and ... all the Algebra I teachers get 
into a room with her. They all take the test together. That's our way of vetting the test. 

They take the test so they can see what’s on the test. The test doesn't leave the room. 

That way we're not teaching the test. But they have an idea of where we're trying to go. 
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Then that happens at the beginning of each nine weeks. And then three weeks before 
we actually administer the test the teachers look at it again and they say to the district 
specialist . . . “That week we had homecoming and a parade and a pep rally . . . and we 
missed three days of instruction over this. And so we didn’t get that far. So that test item 
needs to come out." So we’re working really hard to keep those benchmarks to be a true 
reflection of what we’ve taught. 

Once the tests were given, students’ scanned answer documents from the benchmark assessments 
were stored in the district’s electronic data system, from which teachers and other school personnel 
could produce online reports disaggregated by school, classroom, student, item, and instructional 
objective. Both districts used the same software to report benchmark test results. 

Educators in the two districts described using these reports in a variety of ways. Teachers 
used reports by item or instructional objective to identify objectives that need to be retaught to 
small groups of students or to the entire class. Teachers and school leaders used student-level 
performance reports to place students in intervention. In one conversation, reports by student 
subgroup were used as evidence that the teachers are closing achievement gaps. An educator in 
the same school reported comparing the school's benchmark test results with those of schools in 
the district with more advantaged student populations. In middle schools in District A, students were 
grouped for a one-hour intervention and enrichment program based on their benchmark test results. 
An educator in one school reported comparing students’ grades with their scores on benchmark 
assessments to see if students with passing grades are also passing the district assessment. 

School leaders also reported using benchmark test data to predict how well students were likely to 
do on the state test, while district leaders reported using the benchmark data to see if the district’s 
curriculum scope and sequence was being taught. 

School leaders and academic coaches also reported using benchmark assessment data 
disaggregated by teacher to monitor equity across classrooms: the extent to which student learning 
depends on the teacher to whom the student is assigned. One educator expressed the idea that 
if teachers are addressing the same content and planning their lessons together, students should 
receive similarly effective instruction: 

As a campus specialist I meet with each individual teacher based on their benchmark 
scores. And our goal for the math team, we decided as a team that we didn't want to be 
more than five percentage points away from each other. 

We wanted our range to only be five, because we meet and we plan so often and we 
know what our goals are. Then we felt like if students are getting that equity, the same 
instruction, the same questions, with the teacher’s own twist, then students should be 
able to be successful in any of the classrooms. . . . 

And when a teacher is not within those five points, then as a specialist, I talk to the 
teacher about teaching. What is it? What do you think? And really they already know 
what they missed or what they didn’t do or what they could have done differently. 

Differences in students’ prior academic preparation could also affect comparisons of benchmark 
test scores across classrooms. In neither district did educators report using a formal statistical model 
to adjust for students' prior achievement. Instead, one school leader reported using the prior year's 
student-level state test data to ensure that teachers are assigned a comparable mix of poorly and 
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better prepared students. Another educator reported sharing data on students’ prior year scores with 
teachers to show that students were equitably distributed across classrooms: 

(Prior achievement) was definitely an issue. We had our strong teacher . . . (who) strongly 
believed that they were hand-placing the low students in her class. . . . We looked at 
the data for every student on our campus. And I learned to, instead of just giving (the 
teachers) their own student data, I gave them (data on all the teachers’ students) . . . 
and so they can see their kids and they can see all the other students and kind of say, 
okay, it's not just my students. 

And we have this data binder that has all seventh grade students and how they did as a 
sixth grader on the STAAR (State of Texas Assessments of Academic Readiness— state 
test). And so they see, you don’t just have the low students or ill-prepared students. 

School leaders also reported using benchmark assessment data disaggregated by teacher and 
instructional objective to identify teachers who need additional assistance. According to these 
leaders, test scores by themselves were not treated as a single dispositive source of information 
on a teacher; rather, low scores on benchmark assessments triggered an inquiry and coaching 
process consisting of conversations with the teacher and an increased frequency of classroom 
walkthroughs (structured observations) to diagnose what was occurring in the teacher's classroom. 

If the combination of classroom observations and benchmark assessment data continued to indicate 
that a teacher was not improving despite coaching and support, then the principal might begin the 
documentation process that could lead to the teacher’s dismissal. 

School Common Assessments. In both districts, teachers supplemented benchmark tests 
with “Quick Checks” or “Target Tests” administered and analyzed at the school level to cover 
the content taught in the previous three weeks. In some cases, teachers at each school wrote 
these assessments; in other cases, they were drafted by academic coaches or district curriculum 
specialists and edited or supplemented by teachers at the school level. 

School-level educators reported that these assessments were mainly used to monitor student 
progress, identify content to reteach, identify students in need of assistance, and plan future lessons 
and homework assignments. They were also used, on occasion, to determine whether students had 
mastered skills on which they had performed poorly in an earlier three-week test. At one school, 
teachers set goals for student proficiency on the three-week exam and reviewed their success in 
meeting those targets. Information from these three-week assessments appeared to be used in 
schools and classrooms but was not passed up to district administrators. 

State Accountability Tests. Texas began giving the STAAR tests in grades 3-8 in 2012, 
superseding the Texas Assessment of Knowledge and Skills (TAKS) tests in those grades. STAAR 
end-of-course (EOC) tests were given in five ninth-grade subjects. The STAAR was scheduled 
to replace TAKS in grade 10 in 2013 and grade 11 in 2014. No state accountability ratings were 
assigned in 2012. 

As of the fall of 2012, when the interviews were conducted, the state education agency had not yet 
set scale scores and state proficiency standards on the state test in grades 3-8, so only raw scores 
were available in those grades. A complete longitudinal testing history of each student was available 
from the state for parents and educators on the TexasAssessment.com website. An item analysis on 
students’ incorrect answer choices was not available, since most test items had not been publicly 
released. 
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As with benchmark test results, school leaders and academic coaches reported disaggregating 
and comparing state test results across schools, student subgroups, classrooms, students, and 
learning objectives. Educators used released state test items to indicate the desired level of rigor 
for district benchmark and school common assessments. In one school, teachers reviewed student 
compositions on the state writing test in order to identify common features that distinguished high- 
scoring essays. 

An additional use of state test data was to place students in courses, classrooms, or interventions. 
For example, educators reported using state test scores and student grades to place students in 
middle school pre-AP courses and high school math and science courses. One educator reported 
using the previous year’s state test score data to assign teachers an equitable mix of well- and 
poorly-prepared students. 

College Readiness Tests and Their Precursors. Both districts gave the College Board’s 
ReadiStep exam to all students in eighth grade and the PSAT to all 10th graders, while encouraging 
juniors and seniors to take the SAT. Counselors could use these tests to advise students on their 
college selection and application process. However, school counselors were not interviewed for this 
study. In neither district had the educators interviewed matched the content of these tests with the 
district's written curriculum or used the results from those tests to assess student learning of that 
curriculum. Thus, teachers were not treated as responsible for students’ performance on the test: 

We do college-readiness assessments. And we’re doing one tomorrow . . . and that 
data goes to administrators. It goes to counselors. . . . The test they’re giving tomorrow, 
students haven't prepared for it. . . . It's a requirement. But it’s not anything that teachers 
have learned about or are responsible for. So we don’t use those results to assess 
quality of teaching. 

Other Standardized Assessments. Elementary educators in the two districts reported using a 
number of standardized assessments to monitor student progress in reading and identify students 
who need additional assistance. These assessments included the Texas Primary Reading Inventory 
(TPRI) and its Spanish counterpart, Tejas Lee, in kindergarten through second grade, as well as 
the Writing and Reading Assessment Program (WRAP) in District A and the Diagnostic Reading 
Assessment (DRA) in District B. 

In addition, teachers in both school systems administered the Iowa Tests of Basic Skills (ITBS) in 
reading and mathematics in second grade, and educators used SuccessMaker and other computer 
programs to track their students’ reading and math skills. In District B, educators also used the ITBS 
along with the Cognitive Abilities Test to screen students for gifted and talented programs. In the 
middle grades, educators in District A used the Middle School Students in Texas Algebra Ready 
(MSTAR) Universal Screener, an algebra-readiness test, to place students in tiers for mathematics 
interventions. 

For English language learners, educators in both districts used the state's Texas English Language 
Proficiency Assessment System (TELPAS) to assess student proficiency in written and spoken 
English. In addition, educators in District B administered the Woodcock-Munoz assessment of 
English language proficiency to place or exit students in bilingual programs. One educator reported 
using the TELPAS data to compare the progress of English Language Learner (ELL) students in 
bilingual classrooms with those of ELL students who by parent request had been placed in English- 
only classrooms. 
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Classroom Observations (Walkthroughs). School and district leaders in both districts report 
using classroom walkthroughs to gauge teaching and learning in their schools. Formal walkthroughs 
were required as part of the state’s teacher evaluation process, the Professional Development and 
Appraisal System (PDAS). In both districts, school leaders recorded PDAS observational data on 
tablet computers during the walkthrough; these data were available electronically to the principal 
and teacher afterwards. Outside of the appraisal process, informal walkthroughs were used largely 
to coach and assist teachers. In some cases, academic coaches used walkthroughs to identify 
good practices for discussion in teacher team meetings. Informal walkthroughs were conducted 
more frequently in the classrooms of new teachers and teachers whose students perform poorly on 
assessments: 

Here are your high-performing teachers. They’ve proven that over time with their student 
success data, with their walkthrough data, with their observation data . . . formally we try 
to go into those classrooms once a month. 

Then we look at kind of like our second tier . . . our kind of average teachers who have 
performed proficiently, but we still have some areas that we would like to see them 
grow in. And we’re in their rooms at least twice a month. And then we have a group of 
teachers who we focus on at a much greater level. We have some concerns based on 
student performance data, and we’re in their rooms once a week. 

School leaders also used walkthroughs to keep track of how teachers implemented what they had 
learned in training. For example, teachers in one school were trained on differentiated small-group 
instruction. Walkthroughs were used to monitor how teachers applied this training. School or district 
leaders would sometimes pick a "focus of the week" or "focus of the month” for their walkthroughs 
to concentrate their attention on a specific instructional practice. 

Educators in both districts also used walkthroughs to monitor whether the district's curriculum 
was being taught and to what level of rigor. In District B, district administrators and school leaders 
conducted these walkthroughs together. As one school leader commented: 

We use (classroom walkthroughs) in several ways. If a teacher is having problems, 
of course we're in there a whole lot. And we give immediate feedback which allows 
them to make progress. But we do something called a curriculum walkthrough in this 
district . . . we choose a subject area and we pull out their lesson plans. We pull out our 
(curriculum documents), which are how we divided the curriculum. Does it all match? 

And then we go in the classroom to see if they're teaching the TEKS (Texas Essential 
Knowledge and Skills state standards) that they’re supposed to be teaching at the level 
it’s supposed to be taught ... we watch for a minute, but then we talk to the kids. What 
are you learning about today . . . and is it easy or hard? And why do you need to know 
about this? Where are you going to use this outside of school? 

And so that’s really, I think, beneficial. The teachers all want to know how they did. So it's 
really helped us tighten up our curriculum and see where there are some holes. 

Walkthroughs could also be used to monitor teaching and learning in untested grades and subjects. 
For example, there was no state social studies test until eighth grade, and District A did not give 
social studies benchmark tests prior to fifth grade. When asked how student learning in social 
studies was monitored in the earlier grades, one educator said: 
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Lesson plans, checking their lesson plans to see if there’s evidence of (teaching social 
studies). ... If you take out the daily schedule and just do some walk-throughs when 
they’re supposed to be teaching social studies, that’s a good way to come out and see 
if that implementation is there. 

Student Grades. Each district stored student grades in a separate computer database from the 
one used to hold benchmark and state test scores. (The use of separate databases appeared to 
be driven by the purchase of these systems from separate vendors.) As with test scores, educators 
used grades as an indicator of student learning and as a way to identify students for interventions. 

As administrators, of course, we can look in and see grades . . . and again, we have all 
kinds of data that we can pull up whether you want failing grades, grade distribution, 
whatever that might be. And we use that data . . . when we meet with our departments 
to be able to say to teachers, you had some great success over here. Tell me what you 
did that was successful and let’s keep those things that are successful. 

Over here we struggled a little bit; tell me the things that you tried that maybe weren't 
as successful. What are we going to do? Are we going to remediate? Are we going to 
re-teach? We're going to review? What are we going to do? We just finished that since 
we just finished our first six weeks’ grading cycle. 

In some cases, school leaders examined whether the grade distribution was similar across teachers 
in the same subject. High percentages of students earning failing grades in a teacher’s classroom 
could lead to a conversation with the principal about what was causing the difficulty. One school 
leader mentioned high failure rates as an indicator that the teacher needs additional coaching. 
Educators also compared grades informally with either benchmark or state test scores to make sure 
that students who have good grades are also performing well on assessments. 

Grades were also used in conjunction with test scores to place students in courses— pre-AP courses 
in middle school, and a variety of courses in high school: 

This is all of the freshmen in the school. And I went through individually and looked at 
their scores, how they did in algebra. . . . This is biology, how they did each semester. 

And then this is their algebra EOC (end-of-course exam) grade and their biology EOC 
grade. 

And I looked at each student, almost 1 ,000 of them, and decided if they should be in an 
IPC (Integrated Physics and Chemistry) class, a chemistry class, or some of them had 
to retake biology. Some of them are in pre-AP chemistry. . . . We felt like if they weren’t 
successful in algebra, they’re not going to be successful in chemistry because they 
need those algebraic equations to work the chemistry problems. And so we’re giving 
them that extra support. (In IPC) we’re teaching the basics of chemistry, just not using a 
lot of the formulas ... we use a few of the basic formulas, but not the harder ones that 
they wouldn’t understand the math. 

Student Attendance. In both districts, school leaders reported monitoring daily attendance data, 
and teachers and school counselors were responsible for contacting parents of students who have 
too many unexcused absences. In the area served by each district, the district attorney's office sent a 
warning letter to students’ home addresses if the student had more than three unexcused absences 
in 28 days or six for the year as a whole. District personnel supplied the data needed to generate 
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these letters. For truancy hearings, district personnel also supplied information on the contacts that 
school staff made with the parents. 

Interventions for students with chronic absences were chosen and implemented at the school level. 
One school operated a Saturday camp for students who missed too many classes. In that school's 
neighborhood, absences were often caused by students having to care for sick family members. The 
leader of another school described a program for students who had been chronically absent in the 
previous school year: 

Students who had over 20 absences last year are being monitored either by myself, 
my assistant principal, or my counselor. And we’re working with them on getting them 
to school. . . . One young man had to have surgery so he missed a few days. But that’s 
the only days he’s missed. One young lady's missed four, but everybody else is sitting 
right at zero. So by working with them we’re getting their attendance up. And of course 
that’s resulting in improved grades. I have a spreadsheet that I made that has the kid, 
how many days they missed and what program they’re in, if they’re in one of our special 
programs for remediation. 

And so I was able to show the kids the connection between how many days they missed 
and the remediation program that they’re in that either costs them their elective or will 
cost them their elective in sixth or seventh grade when they leave me. So they’re learning 
about how if I’m not here, I can’t learn what I need to learn. So they're beginning to make 
that connection which is pretty good for a 10, 1 1 -year-old. 

Discipline Reports. In both districts, student-level disciplinary referral data were stored in the same 
database as student grades. In each case, educators in the two districts had the ability to sort the 
data by the type of incident or infraction, the general location in the school, and the action taken in 
response to the infraction. These reports included minor infractions such as being late to class or 
not wearing one's student ID badge as well as more serious rule violations. The great majority of the 
reports were for minor infractions. 

Educators reported using the information to analyze changes between the previous and current year 
in the number of students with different types of infractions, and to identify places and times of the 
day where problems are more frequent: 

We use discipline (data) ... for several things. Is there a hot spot on campus? ... It 
sounds silly, but, okay, the boys' bathroom is one of those places. So now we have a male 
teacher that stands at the door of the boys' bathroom to ensure we don’t have fights in 
there. Because the kids have figured out there’s no video cameras in the bathroom. And 
if they’re going to mess around, that’s where they're going to mess around. 

We'll use discipline data to see if a particular teacher is really struggling. If one teacher's 
referring a lot, a lot of kids, no one else is referring those same kids; it tells us there’s an 
instructional classroom management problem in that classroom. So we work with it that 
way. We look to see if we are meeting and following the BIP (Behavior Improvement 
Plan) for our special needs students. We look to see if there's a trend, if we're, if our 
discipline is appropriate and if there’s a trend, if we have more than, like if there’s— let's 
say our African American students are being referred more than our white kids for the 
same behavior. So we're tracking those kinds of things, making sure we’re fair. 
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A second educator commented that: 


I kind of oversee the discipline process here. So I run reports, for example, sometimes 
if a new student comes in and I know that . . . there's things going on, then I’ll run their 
individual report to see what were some of the challenges they were having. . . . 

Around December, I’ll run a (school-wide) report to kind of see where we’re at. And then 
in May I'll run a final report for the year and we analyze the data and say, okay, how many 
office referrals did we have? What were the consequences that we gave for those office 
referrals? How many of them were level-one documentations? Because we have a level 
system. How many of them were level three? . . . 

And we talk about what we could do differently . . . we've gone through Safe and Civil 
here in our campus, we’re on the third, fourth year of it. And then we’re also going 
through Conscious Discipline. And both of those have really made a huge difference 
with the discipline, because I can already see running reports in December. And thus far 
I have about five discipline referrals for the year. 

One educator used student-level data to look at the relationship between discipline and student 
grades. Another educator used the data to identify students with multiple referrals and had those 
students set personal weekly goals to avoid a further referral in that week. 

Course Completion, Graduation, and Leaver (Dropout) Data. Educators in both districts 
reported monitoring student enrollment in and completion of pre-AP and AP courses. In addition, 
high school leaders in one district reported using course completion (transcript) information to 
indicate whether students are on track to graduate on time, and which students are missing which 
course requirements for graduation. 

High school leaders reported using graduation and leaver data in conjunction with course 
completion information to identify, locate, and recover students who have dropped out. Educators in 
one school discussed the data they use to locate students who drop out: 

We have dropout monitoring reports that we can pull up at any moment. We look at 
them. We sort those withdrawals by leaver code. And those kids that are difficult to find, 
let's say. Of course, we're using TREx (Texas Records Exchange). We're using PEIMS 
(the state’s Public Education Information Management System). We even have a Social 
Security database. We have a couple of other databases that we use to try to locate 
kids. Did we find them all? No, sir, we didn’t. ... So we try to prioritize our leavers so 
that we find those that we have the best chance of finding first, our hot leaves. We’ll find 
them first. And then we’ll work our way through the others. 

Educators in that school conducted an annual “Walk for Success” in the neighborhood to try to reach 
students who have dropped out, using transcript data on each student’s missing requirements to try 
to convince the students to come back and complete them. 

Surveys of Students. Surveys of students or parents in the two districts were usually designed, 
written, and administered by the administrators of each school and were conducted at the discretion 
of the school leadership. At the district level, administrators did not conduct regular annual surveys, 
but instead surveyed occasionally to investigate specific issues. For example, in response to African 
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American middle school students' low mathematics achievement, leaders in District A surveyed the 
students on their relationship with their mathematics teachers. 

Educators in one elementary school surveyed students in second through fourth grades on their 
perceptions about the school, their teachers, and what they were taught: 

The survey with the students, we sat down with our Leadership Team and we went 
over the data. We only surveyed second, third, and fourth. And we talked to our Team 
Leaders about what were some of the things that we were seeing? What were some 
of the concerns? What were some of the celebrations? It was eye-opening. We had 
a large percentage of our second, third, and fourth grade students who believed that 
we weren't teaching higher order thinking. So that hurts a little bit. But that was good 
because then the Team Leaders were able to go back to the teachers and say, “Why 
do you think that the students feel this way?" . . . The way the question (on higher order 
thinking) was asked (was) . . . “My teacher makes me think every day in class” 

Educators in a middle-grades school surveyed students about their preferences for student activities 
and events as well as their perceptions of their teachers, the safety of the school, and their overall 
attitude toward the school. Teachers in a third school collected informal feedback from students and 
compared it with what they had hoped for at the beginning of the year: 

At the beginning of the year before school started I had my teachers write down two 
words they wanted kids to say about their class, two words or two phrases, before 
school ever started. ... At the end of the first nine weeks . . . (the students were asked 
to) write down two words about my class that describes how you feel about my class. 

And so then the teachers took that information to see did what I wanted the kids to say 
about my class match what the kids actually said about my class? . . . And if not, then 
they sat down and tried to figure out how to change what they were doing to change 
the perception of the kids. 

Surveys of Teachers. Some school leaders surveyed teachers for feedback on their leadership— for 
example, whether teachers feel that they are being provided adequate support. In District A, teachers 
at focus meetings were surveyed on which instructional objectives in the previous weeks were the 
most challenging for them to teach, and which ones the students found the hardest to learn. In later 
discussions among the teachers, these perceptions were compared with student performance on 
the objectives on benchmark exams. In addition, curriculum specialists wrote model lesson plans 
to help teachers with objectives that were identified as hard to teach. In District B, administrators 
conducted an annual survey of teachers and school leaders to learn about their expressed needs for 
professional development. 

Surveys of Parents. Parent surveys administered in the two districts addressed issues such 
as overall satisfaction, parent perceptions of school safety, climate, and academics, and their 
perceptions of whether they are welcome at the school and whether the school staff communicates 
well with them. School leaders used the results to identify parents’ concerns and provide information 
in response to those concerns. Educators at one school worked to improve communication about 
school events in response to survey information that parents did not know about many of these 
events. 
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What support did district leaders provide to facilitate the use 
of data by teachers and school leaders? 

School district leaders can play an important role in promoting data use (ACT, 2012; Datnow et 
at., 2007; Kerr et at., 2006; Schildkamp & Kuiper, 2010). District leaders in the two districts took a 
number of steps to facilitate use of data by teachers and school leaders: 

Established a written district curriculum to help teachers plan lessons around the student 
learning objectives that were to be taught in a six- or nine-week period and tested at the end of 
the period. Many of the data reports that teachers use were organized around these objectives. 
Leaders in both districts provided teachers with detailed write-ups of the curriculum in specific 
core courses— viewed as especially helpful for new teachers. Leaders in District B also posted 
the curriculum on the district website so that students and parents, as well as teachers, could stay 
informed about what was being taught throughout the year. 

In both districts, the written curriculum included pacing guides giving the suggested length of time 
to devote to specific topics or objectives. Leaders in both districts allowed for day-to-day flexibility 
in teachers’ implementation of these guides. This was apparent from conversations where teachers 
discussed how long they expected to need to teach each topic. One district administrator described 
the role of the district’s pacing guide: 

If you have a star teacher . . . they can get (what they need) for the year with this year- 
at-a-glance (providing an overview of what is to be taught in each nine-week period). A 
pacing guide is a little bit more guidance and direction for your new or inexperienced or 
new-to-the-district teachers. . . . And obviously ... if they don’t need to spend 10 days 
on mechanical energy, they don’t have to. But (if) they know, oh, brother, space is a hard 
one, they only gave me 15 days but my kids need 20, let them have that flexibility and 
autonomy to know to get done what they need. 

Established a system of curriculum-based benchmark tests to measure student learning of 
the district's written curriculum in the four core subject areas of English language arts, mathematics, 
science, and social studies. 

Developed an electronic information system providing teachers and school leaders with easy 
access to a wide variety of reports disaggregating data in different ways. As one educator described: 

We have Aware in our district which is just a management system for data. And all of 
our state assessments are on there. We can disaggregate that data by SE, or student 
expectation, by the standard. We can disaggregate that data by scale scores. I mean 
we can look at every kid and it will give us a picture of how that child did on that test. 

So we're data rich. Sometimes we don’t know how to use it effectively, but we have it. 

Our benchmarks go on there. Their daily grades can go on there. We have a lot of data. 

I can look at historical test data on every kid on this campus from the third grade on up. 

I know how every kid has done. I have access to that information. So we have access 
to all of that. We can pull up attendance. We can pull up their transcripts, discipline; 
those kinds of reports are readily accessible at least at the administrative level. And the 
teachers' testing information is available on their students that they have in their classes 
this year. They can look at every kid. If I'm in Ms. Jones' third period class she can look 
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(me) up and she can see how I’ve done since third grade on the reading tests, for 
instance, if that’s what she wants to see. 

So data is not a problem. Our teachers have it. We have it. Sometimes we have more 
than we can use, to be honest with you. We’re kind of drowning in it sometimes. So 
figuring out what to use and when to use it is probably our biggest challenge. 

Convened district-wide “focus meetings” of teachers in the same grade and subject area 
to discuss curriculum, instruction, and assessment. In these meetings, teachers of the same 
course-specific content in different schools— for example, Biology 1 or third-grade social studies— 
met to review curriculum and assessments and to share instructional ideas. The timing of these 
meetings was based on the six- or nine-week periods in the district curriculum. The high school 
focus meetings in one district were commonly used to present model lessons for the upcoming six 
weeks' content: 

We have a group of teachers from every school that come together also once every six 
weeks that are called curriculum writers. And we bring them from each school. Like, 
every school has a representative for biology, chemistry, physics, and IPC (Integrated 
Physics and Chemistry). And they get a whole day to spend together to talk about 
what’s worked, what doesn’t work, what they can do to become better. And they all 
bring lessons. They look at those lessons, evaluate them, and see what lessons that we 
need to put on this focus. So that we know whenever we go to a focus meeting, these 
are tried lessons. They’ve worked well in the past. And here’s something you can do to 
help your students. 

Sponsored district-wide vertical team meetings. These meetings brought together teachers 
of the same subject in different grade levels— for example, social studies teachers in K-6, 7-12, 
or K-12. These meetings were held less frequently than the focus meetings— in District A, they 
occurred once a year— and some educators expressed a desire that vertical team meetings be held 
more often. An educator described vertical team meetings in District A: 

They tried to do some vertical meetings by core area, like science. They would have a 
science group and you'd have some maybe from pre-K all the way to 1 2th grade. And 
so they're valuable meetings. And it’s funny to hear the biology teacher saying, “Oh my 
gosh, they learned this in second grade!" They act like they’ve never heard that. “They 
learned about photosynthesis!” . . . That’s happened periodically, not very often because 
it’s hard to get everybody in that room. But it does happen. 

One administrator described District B’s vertical team meetings: 

Fourth and seventh grade social studies teachers meet together, because they both 
teach Texas history. Fifth, eighth, and eleventh grade social studies (teachers) meet 
together because they all teach American history. Sixth science and chemistry meet 
together. So it’s based on which standards feed into which EOC for secondary. 

Hired academic coaches to support teachers with instruction and data use. The hiring of 
academic coaches was a new development in both districts; due to fiscal constraints, neither district 
had been able to hire coaches to cover all core academic subjects in every school. 
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The role of academic coaches differed in the two districts. In District A, academic coaches were 
referred to as “peer facilitators” and were mainly responsible for providing instructional guidance to 
teachers. Two peer facilitators in District A described their role: 

PF1 : The peer facilitator role is individualized to each campus . . . going into classrooms, 
sitting down, working with kids, seeing where the gaps are, modeling a lesson, 
modeling instruction, finding resources, bringing the resources . . . just answering small 
questions. . . . And we really don’t do walkthroughs to gauge a teacher. I mean, we do, 
but we don't do it to gauge a teacher to pass on information (for evaluation). We do it 
to gauge a teacher for our own sitting down working with them. 

PF2: And usually it comes from, like, a teacher saying, “Hey, I’m really struggling here. 
Can you walk through my class and see what I’m doing to see if it’s working or not 
working?” 

PF1: Every teacher teaches the same lesson differently. And so we walk through to 
see is it being taught at the rigor that we want. Is it getting across to the kids? Are 
the students engaged? . . . Because then if it's a no, then we know we need to start 
modeling more for that teacher, start moving in, start making sure that they’re following 
the high level lesson plan that we've laid out as a team. 

In District B, in addition to providing instructional guidance to teachers, academic coaches were 
responsible for providing direct academic assistance to students. Accordingly, the policy in District 
B was for the coaches to spend at least 70% of their time working directly with students and the 
remaining time advising teachers. As one educator in District B explained: 

(Academic coaches) have two roles. One of the roles is first of all, they’re master 
teachers. And they use student data to help guide instruction. They work with small 
groups of students. On our campus they have one class that belongs to them. It's our 
lowest of the low kids . . . and it’s an hour and a half-long class, so they have 45 minutes 
of reading and 45 minutes of math, and they switch. So my bottom 20 fifth graders and 
my bottom 20 sixth graders are in there. . . . 

And then the rest of the time they float among teachers wherever there's a need. But 
they also have, in addition to working with kids, they also are beginning to work with 
teachers on improving instruction. They’ve always said, “Have you thought about..." or 
“Hey, I saw this work in so-and-so’s class.” But now they’re beginning to take a more 
direct role. 

It's a difficult switch for them because they're not really supervisors, they're still teachers. 

It’s a difficult switch for the teachers because you’ve got another teacher telling you, you 
need to make some changes. So they’re handling it with finesse. They're becoming 
more (like) instructional coaches. 

Encouraged school leaders to promote teacher collaboration around data use. At the time 
of the interviews, school leaders in both districts were participating in a district-organized study of 
a book on building a partnership and coaching relationship among teachers, academic coaches, 
and school leaders. Both districts had encouraged school leaders to provide time for teachers to 
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meet during working hours to discuss curriculum, instruction, and student assessment data. In both 
districts, scheduling of these planning times was arranged at the school level. 

One district administrator commented that a district goal was to develop an effective professional 
learning community (PLC) among principals across the district, giving those leaders a better 
understanding of how those communities should work in their own schools. A second district 
administrator described the district's efforts to promote data use among school educators: 

What we’re working with currently ... is the concept of PLCs and Data Teams. So we’ve 
been working with Unmistakable Impact, Jim Knight’s work as well as DuFour on how to 
create those teams at their campuses. I think one easy read that we used at Principals’ 
Academy was the data teams (the book by Angela Peery, The Data Team Experience). 

And that’s really focusing on, okay, we take a CBA, we get the results back, now what? 

How does that drive instruction? So we’ve talked a lot to not only administrators, but 
they’ve taken that down to the teacher level as well as our awesome Staff Development 
Department has worked really hard to help teachers understand what they’re looking 
at, how can that drive instruction, what they need to do with it, what the next steps are, 
what do those conversations look like, what they don’t look (like)— so I think those are 
some of our initial efforts. It’s part of our strategic plan for the district to have PLCs on 
all of our campuses. 

Encouraged school leaders to coach their teachers on data use. Most teacher professional 
learning on the use of data in the two districts appeared to occur in focus meetings or in the setting 
of their own schools in collaboration with their colleagues and with school leaders. Principals, 
assistant principals, and academic coaches were expected to help teachers become more proficient 
in data use. In this regard, one district leader commented on the importance of school leaders 
coaching teachers on how to analyze student data: 

What our teachers need is time to learn how to make (the data) their own. . . . (Some 
of our principals) are outstanding at data analysis. And they love it. And they can get it; 
they can analyze and interpret it. And that’s the problem. They gave it to their teachers 
already interpreted and analyzed and already basically applied. . . . Well then that just 
goes in a folder or in a drawer or maybe it’s stuck in their lesson plans. But if they don't 
own it they’re not going to do anything different with it. So when those . . . principals got 
their CoreWork data back and all of the teachers said they need help analyzing data, 
they were floored. They were like, ‘‘Oh, that’s my strength! That’s what I do!" And it was 
a big ah-ha moment, like, yeah, that's what you do. But you’re not teaching them how 
to do it. 

What support did school leaders provide to facilitate the use 
of data by teachers and students? 

School leaders can play an important role in promoting data use (ACT, 2012; Schildkamp & Kuiper, 
2010; Wayman, Spring, Lemke, & Lehr, 2012). School leaders in the two districts had taken several 
steps to facilitate use of data by teachers: 
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Worked to create a climate of trust among their teachers. This study did not use any direct 
measures to assess trust. However, the teacher team meetings that were observed proceeded with 
a tone of relaxed collegiality that would not have been possible without a degree of frust among the 
teachers, academic coaches, and school administrators who were present. 

One school leader described the challenge of creating an environment in which student results and 
ideas for improvement can be freely shared: 

(We are) trying to establish ... a professional learning community where people are 
okay having those kinds of critical conversations. And it’s not a personal thing if I tell 
you that you could do this a little bit better. Or you could change some things. ... So 
as we’ve kind of grown into this PLC, it's like when we're having these professional 
conversations it’s only causing us to grow and stretch. It's not a personal attack on who 
you are as a teacher. That’s not it. 

And the more and more it becomes part of our culture, the less and less people are 
offended by it. And they don’t feel like you’re picking on me. . . . It's just part of what we 
do is we have critical conversations. We talk about things and maybe we disagree. And 
that’s okay. 

Arranged the master schedule to have common planning times for teacher teams. In both 
districts, creation of master schedules was a school-level responsibility. All of the schools visited 
had established common planning times for teacher teams as part of the school's regular routine. In 
one school, the master schedule was posted on a wall with the common planning times for teachers 
clearly shown. One school leader described the importance of these planning periods: 

We have created common planning times for all of our teams so that they can get 
together and talk about instruction and share good ideas, share teaching strategies 
and sort of work together on finding the best of what everybody’s doing and replicating 
that everywhere. So that just because you’re assigned to my class you don't struggle. 
Because I may be the weakest link in the Algebra I team, (but) I’m getting the benefit of 
all the teachers and all the expertise. 

We just started this year. We rearranged the master schedule. For instance, all of my 
Algebra I teachers have first period off. And they plan together every day, or at least 
several times a week. All of geometry has fourth period off, so forth and so on. So we 
fixed the master schedule where content teams have a common planning time during 
the school day. So it’s not all having to be done after school. That helps a lot. Because 
after school they're tired. And so it helps when they can at least get the process started 
during the regular day. So we’ve been proud of being able to do that. 

Met with teachers to discuss their students’ academic results. In one school, the principal met 
one-on-one with each teacher of a core academic subject (reading/writing, mathematics, science, 
and social studies) every nine weeks to discuss their students' benchmark test results. Another 
principal described holding diagnostic conversations with teachers whose students performed badly 
on those tests, while delegating these conversations to academic coaches or department heads 
when the results were better. In some cases, principals led the data discussion in teacher team 
meetings; in others, that leadership was delegated to teachers or academic coaches. 
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Encouraged teachers and students to set goals based on data. Teachers at one school 
set goals for their students' performance on common assessments and reviewed whether those 
goals were met in a subsequent meeting. In another school, students set improvement goals for 
themselves on three-week common assessments. An academic coach in that school described how 
students were responsible for keeping track of their own progress: 

It's been pretty amazing to hear the students talk about their own in-depth data. The 
first student-led conference that we had was pretty early on in the year. And so they 
had just had their STAAR data from the previous year. But you hear them having those 
conversations. Well the district was at this. But I was here. Now let’s look at my individual 
classes. I’ve only done one target test or one Quick Check but this is what I did. 

And (the students said) constantly, these were my goals and I met this goal and then I'm 
into this goal. And I haven’t met this goal yet. . . . So, those three questions on fractions 
or decimals or equivalent fractions, I missed two out of the three. So I know this is a low 
area for me. ... So we're really kind of putting it in the hands of the students as well as 
the teachers (to) know individually what (they) are strong and weak in. 

What information was discussed in teacher team meetings 
and how was it interpreted? 

At one time or another in the seven school-level teacher team meetings and two district-wide focus 
meetings that were observed, teachers: 

Reviewed items on common or benchmark assessments to identify specific skills with 
which students were having difficulty. For example, the ninth-grade English teachers discussed 
common assessment items that were missed by high percentages of the students. Teachers in the 
meeting had a copy of the exam, a table of the skills covered by each question, and the percentage 
of students who got each item correct. Items missed by more than 30% or 60% of students were 
color-coded. The teachers discussed explanations for why students missed the items and ideas for 
how to improve the students' skills. Likewise, the seventh- and eighth-grade science and seventh- 
grade mathematics teachers went over an item analysis from the district's benchmark assessment 
and discussed why students had difficulty on specific test questions. 

Reviewed and edited benchmark assessment items. The third-grade social studies teachers 
in the district-wide focus meeting reviewed two previously administered benchmark social studies 
tests. The teachers were provided with an item analysis on the percentage of students who chose 
each right or wrong answer. The teachers discussed reasons why students might have missed the 
items and suggested edits to the items’ text and graphics. 

Compared student performance on common and benchmark assessments measuring 
the same skills. The seventh- and eighth-grade science teachers looked over a sheet comparing 
student performance on the same skills on two different assessments. One teacher commented that 
the results were limited by the small number of test questions per skill on each assessment. 

Discussed whether they had met goals they had set for the percentage of students 
mastering specific student expectations. In meetings of second- and third-grade mathematics 
teachers, the principal reviewed how the students in the different classrooms had performed on 
the most recent common assessment relative to their performance on a previous assessment, and 
relative to goals the teachers had set. 
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Discussed what curricular objectives the team would have time to teach in the next few 
weeks. The ninth-grade biology teachers discussed the topics that they would be able to teach prior 
to the next benchmark assessment. One of the teachers had been delegated to write the upcoming 
six-week benchmark assessment, and that assessment needed to align with what everyone would 
have time to teach. 

Discussed vocabulary and prior knowledge relevant to the objectives currently being 
taught. The ninth-grade biology teachers discussed what chemistry terms and concepts students 
appeared to have learned in middle school, in comparison with the knowledge assumed in test 
items from the previous benchmark assessment. The third-grade social studies teachers discussed 
whether students should be familiar with words such as “cobblestones” and “trailblazer” used on 
prior benchmark assessments, and whether third graders should be explicitly taught the names of 
the three branches of government— legislative, executive, and judiciary— as those terms had been 
used on a previous third-grade benchmark test. 

Discussed lessons learned from reviewing student work. One English teacher discussed the 
results of her review of essays that earned the highest score on the previous year's STAAR test, 
describing her conclusion about what made those essays stand out. 

Discussed strategies for teaching curricular objectives. One biology teacher discussed a way 
of introducing the immune system to students. The third-grade math teachers discussed when it was 
important to get students to apply specific strategies they had been taught and when they should 
be expected to show their work on daily assignments and common assessments. The seventh- 
grade mathematics teachers discussed strategies for teaching students to solve word problems 
that involve fractions. One teacher in the prekindergarten and kindergarten English language arts 
meeting discussed her experiences introducing students to Writers’ Workshop. 

Discussed instructional resources that could be used to teach the curricular objectives. 

For example, the third-grade social studies teachers mentioned a variety of instructional resources 
for teaching about individuals and events in the social studies curriculum that were tested on the 
benchmark assessments. The ninth-grade English teacher reviewing high-scoring student essays 
provided information on resources to teach the skills reflected in those essays. 

Conclusion 

The interviewees and teacher team members who were observed provided many examples of data 
use that are likely to benefit students. Several opportunities for improving data use may be inferred 
from the interviews and observations in this study: 

Teachers and school leaders can encourage students to use data more frequently to track 
their own performance and set personal goals. In most cases, grades are the main information 
students use to keep track of their own academic progress. However, the use of district-created 
benchmark exams and school-created common exams creates additional opportunities for students 
to monitor their own progress and set personal academic goals. 

District leaders can increase the frequency of vertical team meetings across elementary 
and secondary levels. One issue in both districts had been coordinating the curriculum between 
the elementary-middle (K-6) and secondary (7-12) levels. District leaders can encourage this 
coordination by establishing more opportunities for members of vertical subject area teams to meet. 
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District leaders can make available reports on student achievement organized by the 
school the students previously attended. For example, data reports could provide information 
on how the graduates of each elementary school perform in middle school and how the graduates 
of each middle or intermediate school fare respectively in intermediate and high school. This can 
help educators in earlier grades know whether they have prepared students well to succeed in 
later grades. This information could be paired with information for educators in upper grades on the 
success of students in each of their earlier grade feeder schools on specific instructional objectives. 
The combination of the two types of information could facilitate vertical team conversations about 
curriculum and instruction across elementary and secondary school levels. 

At the time of the interviews, educators were doing this analysis by hand. For example, one middle 
grades educator commented that: 

When we found out they (the receiving high school) were the highest in biology out of 
all the high schools in our district, (our science specialist) began looking at who passed 
and identifying . . . which ones were our kids out of all the ninth graders. And we were 
able to do that manually. But I'm sure there are ways that we can track a student. But it’s 
probably coming, but we’re not there yet. 

District leaders can encourage the use of data for predictive analysis. Educators, school 
district statisticians, and independent researchers can analyze which student outcomes in the 
early grades— e.g., assessment performance, attendance, discipline, and student engagement— are 
associated with desired outcomes in the upper grades (Sawyer & Gibson, 2012; ACT, undated). 

This can lead to better targeting of long-term intervention and prevention efforts (ACT, 2013). The 
availability of student data in longitudinal databases should facilitate this type of analysis. 

In general, data use should be thought of as one component of an overall long-term system of 
improving teaching and learning that includes clarifying curriculum and academic goals; improving 
staff selection, leadership, and capacity building; selecting and using appropriate instructional 
tools, programs, and strategies; monitoring performance and progress; and identifying appropriate 
interventions and adjustments (ACT, 2012). 
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Appendix A 


Questions for District Leaders 

Use of Data to Address Questions about Teaching and Learning 

1 . Which of the following kinds of information do you use to assess the quality of teaching and 
learning in the schools in this district? 

a) Information on student learning outcomes, behavior, and engagement: 

■ State assessments 

■ District benchmark assessments 

■ College readiness assessments 

■ Other district-purchased assessments (used for what purposes?) 

■ Assessments developed by teachers 

■ Student attendance 

■ Completion of regular courses 

■ Completion of advanced courses 

■ Student grades 

■ Discipline reports 

■ Student graduation 

■ Evidence of student success after they graduate 

■ Student participation and success in academic competitions 

■ Other (please specify) 

b) Information on participant satisfaction: 

■ Surveys of students 

■ Surveys of teachers 

■ Surveys of parents 

c) Information on teaching and learning processes: 

■ Classroom walkthroughs (use an instrument?) 

■ Reviews of assignments given to students 

■ Other (please specify) 

2. What kinds of information do you use to keep track of the quality of teaching and learning in 
grades not tested by the state, in each of the following four content areas? 

■ reading/English language arts 

■ mathematics 

■ science 

■ social studies 
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3. For which of the following kinds of information do you have convenient access to reports? 
(Please provide copies or screenshots of the reports minus any confidential student 
information.) (Do teachers and school leaders also have access? How timely are these 
reports? How do you keep track of how they are used?) 

4. How does your district define being on track to graduate high school ready for college and 
career? 

5. What kinds of data do you use to determine whether students are on track to graduate high 
school ready for college and career? 

6. Which kinds of information are used on an ongoing basis to make adjustments to teaching and 
learning? 

7. What kinds of information do you find most useful for long-term improvement planning? 

8. Can you give examples of how you have used one or more of these types of information in 
making a decision or advocating a course of action for school improvement? 

9. Is there any body of research or set of research findings that you have found useful in your 
efforts to improve teaching and learning? 

10. What kinds of information do you wish you had more of? 

11 . Is there any kind of information that you wish you had less of? 

12. What kinds of information do you routinely share with external audiences (School Board, 
business and community leaders) to keep them informed about student learning in your district? 

Facilitating the Use of Data by Teachers and School Leaders 

13. What kinds of support in each of the following areas have you provided to teachers and school 
leaders to facilitate their use of data to improve teaching and learning? 

■ Training 

■ Scheduled meeting time during working hours 

■ Meetings with district leaders or supervisors (how often; focus of meetings) 

• Online reports 

■ Vertical team meetings across elementary, middle, and high school levels 

■ Other support (specify) 

14. What kinds of information or documentation is readily available to provide more detail on each 
kind of support? 
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Questions for School Leaders 

Use of Information to Address Questions about Teaching and Learning 

1 . Which of the following kinds of information do you use to assess the quality of teaching and 
learning in your school? (Some may not apply to schools at your level.) 

a) Information on student learning outcomes, behavior, and engagement: 

■ State assessments 

■ District benchmark assessments 

■ College readiness assessments 

■ Other district-purchased assessments (used for what purposes?) 

■ Assessments developed by teachers 

■ Student attendance 

■ Completion of regular courses 

■ Completion of advanced courses 

■ Student grades 

■ Discipline reports 

■ Student graduation 

■ Evidence of student success after they graduate 

■ Student participation and success in academic competitions 

■ Other (please specify) 

b) Information on participant satisfaction: 

■ Surveys of students 

■ Surveys of teachers 

■ Surveys of parents 

c) Information on teaching and learning processes: 

■ Classroom walkthroughs (use an instrument?) 

■ Reviews of assignments given to students 

■ Other (please specify) 

2. What kinds of information do you use to keep track of the quality of teaching and learning in 
grades not tested by the state, in each of the following four content areas? 

■ reading/English language arts 

■ mathematics 

■ science 

■ social studies 

3. For which of these kinds of information do you have convenient access to reports? (Please 
provide copies or screenshots of the reports minus any confidential student information.) 

4. How does your district define being on track to graduate high school ready for college and 
career? 
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5. What kinds of data do you use to determine whether students are on track to graduate high 
school ready for college and career? 

6. Which kinds of information are used on an ongoing basis to make adjustments to teaching and 
learning? 

7. What kinds of information do you find most useful for long-term improvement planning? 

8. Can you give examples of how you have used one or more of these types of information in 
making a decision or advocating a course of action for school improvement? 

9. Is there any body of research or set of research findings that you have found useful in your 
efforts to improve teaching and learning? 

10. What kinds of information do you wish you had more of? 

11 . Is there any kind of information that you wish you had less of? 

12. What kinds of information do you routinely share with external audiences (e.g., parents, local 
business and community supporters) to keep them informed about student learning in your 
school? 

Facilitating the Use of Data by Teachers and Other Staff 

13. What kinds of support in each of the following areas does your district routinely provide or 
arrange to facilitate the use of data by you and by teachers in your school? 

■ Training 

■ Scheduled meeting time during working hours 

■ Meetings with district leaders or supervisors (how often; focus of meetings) 

■ Online reports 

■ Vertical team meetings across elementary, middle, and high school levels 

■ Other support (specify) 

14. What kinds of information or documentation is readily available to provide more detail on each 
kind of support? 
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List of Codes Used for Interview Analysis 

academic coaches 

discipline 

academic competitions 

district benchmarks 

adjustments to teaching 

documents to request 

administrator supervision 

double blocking 

administrator team meetings 

electronic reports 

assignments 

entering students 

behavior programs 

extracurricular activities 

CCR (college and career readiness) data 

focus meetings 

CCR definition 

followup data 

college and career planning 

formative assessment 

college readiness tests 

grades 

common assessments 

graduation 

CoreWork Diagnostics 

horizontal team meetings 

course completion 

instruction 

course placement 

instruction by SE 

curriculum pacing 

instructional resources 

curriculum planning 

intervention selection 

curriculum presentation 

item analysis 

curriculum specialists 

less information 

curriculum writing 

lesson planning 

data analysis 

long-term planning 

data by school 

longitudinal student data 

data by SE (student expectation) 

master schedule 

data by student 

mathematics 

data by subgroup 

more information 

data by teacher 

other standardized tests 

data conferences 

parent communication 

data informed improvement decision 

parent survey 
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peer observations 
piloting new programs 
pre-assessment 

principal PD (professional development) 

prior knowledge 

public communication 

reading 

research base 

science 

social studies 

special ed referral rates 

state performance standards 

state tests 

student attendance 

student data use 

student survey 


subject area specialization 
teacher attendance 
teacher knowledge 
teacher PD 
teacher supervision 
teacher survey 
teacher team meetings 
teacher turnover 
test alignment 
test score inflation 
untested subjects 
vertical alignment 
vertical team meetings 
vocabulary 
walkthroughs 
writing 
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